Visual sensor fusion and data sharing across connected vehicles for active safety

ABSTRACT

By exchanging basic traffic messages among vehicles for safety applications, a significantly higher level of safety can be achieved when vehicles and designated infrastructure-locations share their sensor data. While cameras installed in one vehicle can provide visual information for mitigating many avoidable accidents, a new safety paradigm is envisioned where visual data captured by multiple vehicles are shared and fused for significantly more optimized active safety and driver assistance systems. The sharing of visual data is motivated by the fact that some critical visual views captured by one vehicle or by an infrastructure-location are not visible or captured by other vehicles in the same environment. Sharing such data in real-time provides an invaluable new level of awareness that can significantly enhance a driver-assistance, connected vehicle, and/or autonomous vehicle&#39;s safety-system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/620,506, filed Jan. 23, 2018. The entire disclosure of the aboveapplication is incorporated herein by reference.

FIELD

The present disclosure relates to visual sensor fusion and data sharingacross vehicles for improved safety.

BACKGROUND

Vehicle-accident related fatalities, especially those caused by humanerrors exceed more than one million every year worldwide. In response tosuch statistics, a variety of safety measures have been proposed. Inparticular, in the United States, the US Department of Transportation(USDOT) in collaboration with state-level DOTs and experts nationwidehave pursued the development of the Dedicated Short-Range Communications(DSRC) technology and related standards, which are designed forsignificantly improving safety measures through vehicle-to-vehicle (V2V)and vehicle-to-infrastructure (V2I) communications. The USDOT pilot testprogram concluded that DSRC can reduce vehicle related accidentsignificantly. The USDOT also issued a recommendation that the DSRCtechnology should be mandated for all new light vehicles in the nearfuture.

One important category of vehicle-related accidents involvespedestrian-vehicle collision. In the US in 2015, the number ofpedestrian fatalities caused by vehicle accidents was 5,376, a 23%increase from 2009. Pedestrians' fatality is one of the few categoriesthat experienced an increase in the past few years. Furthermore, most ofthe pedestrian accidents happen in urban areas.

One of the many accident scenarios that involve pedestrians is when astopping vehicle occludes a crossing pedestrian from being viewed byother vehicles. A second passing vehicle's driver only notices thepresence of a crossing pedestrian after the pedestrian is within a veryclose proximity to the second vehicle as shown in FIG. 1. In suchscenario, the passing vehicle driver may fail to stop the vehicle in atimely manner, due to the close proximity to the pedestrian, and thisleads to a potential injury or even fatality for the pedestrian.

A variety of new vehicle models typically include an Advanced DriverAssistant System (ADAS) that helps prevent pedestrian and other forms ofaccidents. The success of such system usually depends on the distancebetween the moving vehicle and pedestrian and on the vehicle speed.

This section provides background information related to the presentdisclosure which is not necessarily prior art.

SUMMARY

This section provides a general summary of the disclosure, and is not acomprehensive disclosure of its full scope or all of its features.

A method is presented for sharing data across vehicles for improvedsafety. In a transmitting vehicle, the method includes: detecting anobject in an image captured by an imaging device in a transmittingvehicle; determining a first location of the object from the image,where the first location of the object is defined with respect to thetransmitting vehicle; sending the first location of the object from thetransmitting vehicle via a dedicated short range communication link to areceiving vehicle. In the receiving vehicle, the method includes:receiving the first location of the object from the transmittingvehicle; determining a vehicle location of the transmitting vehicle withrespect to the receiving vehicle; determining a second location of theobject using the first location and the vehicle location, where thesecond location is defined with respect to the receiving vehicle; andimplementing a safety measure in the receiving vehicle based on thesecond location of the object.

In one embodiment, the object is detected using the You Only Look Once(YOLO) object detection algorithm.

The first location of the object further can be determined bycalculating a distance to the object by

$l = {f_{c}\frac{R_{h}}{I_{h}}}$

where the object is a person, f_(c) is the focal length of the imagingdevice, R_(h) is actual height of the person and I_(h) is height of theperson in image pixels. In some instance, the first location of theobject is sent from the transmitting vehicle only if the distancebetween the object and the transmitting vehicle is less than apredefined threshold. In other instance, the first location of theobject is sent to from the transmitting vehicle to the receiving vehiclewhen the two vehicles are traveling in the same direction.

Example safety measure include but are not limited to issuing a warningabout the object to a driver of the receiving vehicle, displaying theobject to the driver of the receiving vehicle or automatically brakingthe receiving vehicle.

In some embodiments, the method further includes capturing, by a cameradisposed in the receiving vehicle, video of a scene; receiving the imagedata for the object from the transmitting vehicle; fusing the image datafor the object into the video; and presenting the video with the imagedata fused therein to the driver of the receiving vehicle.

A collision avoidance system is also presented. The system includes: afirst camera, a first image processor and a first transceiver disposedin a transmitting vehicle. The first image processor is configured toreceive image data from the first camera and operates to detect anobject in the image data and to determine a first location for theobject from the image data, where the first location is defined withrespect to the transmitting vehicle. The first transceiver is interfacedwith the first image processor and sends the first location for theobject via a wireless communication link to a receiving vehicle.

The system also includes a second transceiver and a second imageprocessor in the receiving vehicle. The second transceiver is configuredto receive the first location of the object from the transmittingvehicle. The second image processor is interfaced with the secondtransceiver, and operates to determine a vehicle location of thetransmitting vehicle with respect to the receiving vehicle and todetermine a second location of the object using the first location andthe vehicle location, where the second location is defined with respectto the receiving vehicle. In some embodiments, the second imageprocessor implements a safety measure in the receiving vehicle based onthe second location of the object.

In an example embodiment, the first location of the object istransmitted from the transmitting vehicle to the receiving vehicle inaccordance with Dedicated Short-range Communication (DSRC) protocol.

The transmitting vehicle may also send image data for the object via asecondary communication link that differs from the primary wirelesscommunication link between the vehicles.

The collision avoidance system may further include an automaticemergency braking system in the receiving vehicle, wherein the secondimage processor operates to automatically braking the receiving vehiclebased on the second location of the object.

In some embodiments, the receiving vehicle includes a camera, such thatthe second image processor receiving video from the second camera, fusesthe image data for the object with the video, and presents the videowith the image data to the driver of the receiving vehicle.

Further areas of applicability will become apparent from the descriptionprovided herein. The description and specific examples in this summaryare intended for purposes of illustration only and are not intended tolimit the scope of the present disclosure.

DRAWINGS

The drawings described herein are for illustrative purposes only ofselected embodiments and not all possible implementations, and are notintended to limit the scope of the present disclosure.

FIG. 1 is a picture of an example pedestrian collision scenario;

FIG. 2 is a diagram of a collision avoidance system;

FIG. 3 is a flowchart illustrating an example process for sharing databy a transmitting vehicle;

FIG. 4 is a flowchart illustrating an example process for fusing data bya receiving vehicle;

FIG. 5 is a schematic of an example collision scenario;

FIG. 6 is a diagram depicting pin hole model and image transposecalculations;

FIG. 7 is a graph illustrating bandwidth between two DSRC units;

FIG. 8 is a graph illustrating packet delay between two DSRC units;

FIG. 9 is a graph illustrating various delay in the proposed collisionavoidance system; and

FIG. 10A-10F are an example of fused images shown to the driver of thecollision avoidance system.

Corresponding reference numerals indicate corresponding parts throughoutthe several views of the drawings.

DETAILED DESCRIPTION

Example embodiments will now be described more fully with reference tothe accompanying drawings.

FIG. 2 depicts an example of a collision avoidance system 20. Thecollision avoidance system 20 is deployed in across vehicles. In thisexample, the collision avoidance system 20 is operational between atransmitting vehicle 21 and a receiving vehicle 22. Each vehicle 21, 22is equipped with an imaging device 23, an image processor 24, and atransceiver 25. The vehicles may also be equipped with otherconventional vehicle subsystems, including but not limited to a vehiclenavigation system with a display 26 as well as an automatic emergencybraking system 27, such as the Pedestrian Collision Avoidance System(PCAS). More or less vehicles may be equipped in a similar manner andcomprise part of the system. In some embodiments, designatedinfrastructure-locations, such as signs, traffic signals, bridges, etc.,can also be equipped in a similar manner and comprise part of the system20.

In the example embodiment, the imaging device 23 is a camera integratedinto a vehicle. The system can be extended to employ any sensor modalityincluding lidars, radars, ultrasonic sensors, etc. A more powerfulsystem can be realized by the fusion of a multimodal-sensor system suchas any combination of cameras, lidars, radars, and/or ultrasonicsensors. In cases of sensor modalities that generate a large amount ofdata, the need for data compression could become necessary. Hence, inthe case of using visual sensors, video compression/decompression willbe critical for achieving efficient communication among the vehiclesand/or infrastructure. Any state-of-the-art video coding standards ortechnology that is either standalone or built-in within popular camerascan be used.

In an example embodiment, the image processor 24 is a Nvidia Drive PX 2processor. It should be understood that the logic for the control ofimage processor 24 can be implemented in hardware logic, software logic,or a combination of hardware and software logic. In this regard, imageprocessor 24 can be or can include any of a digital signal processor(DSP), microprocessor, microcontroller, or other programmable devicewhich are programmed with software implementing the above describedmethods. It should be understood that alternatively the controller is orincludes other logic devices, such as a Field Programmable Gate Array(FPGA), a complex programmable logic device (CPLD), or applicationspecific integrated circuit (ASIC). When it is stated that imageprocessor 24 performs a function or is configured to perform a function,it should be understood that image processor 24 is configured to do sowith appropriate logic (such as in software, logic devices, or acombination thereof).

In the example embodiment, the wireless network between vehicle is basedon underlying DSRC transceivers 25 that adhere to the IntelligentTransportation System of America (ITSA) and 802.11p WAVE standards, andwhich are certified by the US DOT. By default, DSRC equipmentperiodically sends Basic Safety Messages (BSM). The messages containvehicle status and applications information. DSRC is merely illustrativeof how a wireless data link may be established between vehicles andother communication protocols fall within the broader scope of thisdisclosure.

FIG. 3 illustrates an example process for sharing data by a transmittingvehicle. Image data is captured at 31 using an imaging device in thetransmitting vehicle. Image data may be captured continuously,periodically or in response to a trigger signal. In the exampleembodiment, the imaging device is a camera although other types ofimaging devices are contemplated by this disclosure.

Image data is then analyzed at 32 to detect and/or identify objects ofinterest, such as a pedestrian, another vehicle or other potentialhazards. In an example embodiment, objects are detected using a You OnlyLook Once (YOLO) object detection algorithm. For further detailsregarding YOLO object detection, reference may be had to “YOYLO9000:Better, Faster, Stronger” ArXiv:1612.08242 December 2016 which isincorporated by reference. It is readily understood that other objectdetection methods also fall within the scope of this disclosure.

Next, a determination is made regarding whether to share data about thedetected object with other vehicles. In this regard, the location of theobject is determined at 33 from the image data. This first location ofthe object is defined with respect to the location of the transmittingvehicle. That is, the transmitting vehicle serves as the reference framefor this first location. Techniques for determining a distance to anobject from the imaging data is readily known in the art. For example,when a vehicle detects a pedestrian crossing, it estimates thepedestrian distance l as follows:

$\begin{matrix}{l = {f_{c}\frac{R_{h}}{I_{h}}}} & (1)\end{matrix}$

where f_(c) is the focal length and R_(h) and I_(h) are the realpedestrian height in meter and height in image pixels, respectively.

Two different criteria are applied before sharing object information,including its location, with nearby vehicles. First, a criterion may beapplied to determine whether a nearby vehicle is a vehicle of interest(i.e., a vehicle to which the object information is to be sent to) asindicated at 34. An example criterion is that object information shouldonly be sent to vehicles located next to or behind the transmittingvehicle. Vehicles in front of the transmitting vehicle are not ofinterest and will not be sent object information. Other example criteriaare that vehicles of interest should be traveling in the same directionas the transmitting vehicle and/or should be no more than two lanes awayfrom the transmitting vehicle. Other types of vehicle criteria arecontemplated by this disclosure.

Second, a criterion is applied to determine whether the object is ofinterest to the recipient vehicle as indicated as 35. For example, onlyobjects within a predefined distance (e.g., I<50 meters) from thetransmitting vehicle are deemed to be objects of interest. Objectsfalling outside of the predefined distance are not of interest andinformation about these objects will not be shared with other vehicles.Likewise, other types of object criteria are contemplated by thisdisclosure.

For each vehicle of interest, object information is sent at 36 via awireless data link from the transmitting vehicle to the vehicle ofinterest (i.e., receiving vehicle). In an example embodiment, thewireless network is based on underlying DSRC transceivers that adhere toIntelligent Transportation System of America (ITSA) and 802.11p WAVEstandard. In this case, object information is transmitted periodicallyusing Basic Safety Messages (BSM) over the DSRC link. Again, it is onlynecessary to send information for objects of interest.

Furthermore, image data for an object of interest (e.g., video segment)is sent to the vehicle of interest. To do so, the transmitting vehicleestablishes another secondary data connection between the transmittingvehicle and the receiving vehicle. In one example, the transmittingvehicle may establish a TCP connection with the vehicle of interest.Rather than sending all of the captured image data, the transmittingvehicle can send only data corresponding to the object of interest. Forexample, the transmitting vehicle sends the image data contained in aboundary box that frames the object as designated by the objectdetection algorithm. Prior to sending the image data, the image data ispreferably compressed as indicated at 37. For example, the image datacan be compressed using a compression algorithm, such as Motion JPEG.Different types of compression methods fall within the broader aspectsof this disclosure. In any case, the image data for the object is sentat 38 by the transmitting vehicle to the receiving vehicle. It is to beunderstood that only the relevant steps of the processing by the imageprocessor 24 are discussed in relation to FIG. 3, but that othersoftware-implemented instructions may be needed to control and managethe overall operation of the system.

FIG. 4 illustrates how shared data is processed by a receiving vehicle.Table 1 defines the variables that are used in system parametercalculations set forth below.

TABLE 1 A Vehicle A B Vehicle B C Pedestrian D Expected collision pointw Vehicle width d Vertical distance between vehicle A and B (similar toΔZ) l Distance between vehicle B and pedestrian e Horizontal distancebetween vehicle A and B (similar to Δχ) r Horizontal distance betweenpedestrian and vehicle B α Angle between vehicle A and pedestrian βAngle between vehicle B and pedestrian n Euclidian distance betweenvehicle A and pedestrian k Euclidian distance between vehicle B andpedestrian ΔY Difference between camera A and camera B altitudeThe reported locations could be measured in any distance units. Forexample, they could be in meters as used in the Universal TransverseMercator (UTM) coordinate format. Also, the camera location isconsidered as a vehicle reference location. If more than one pedestrianis detected, the same calculations can be performed for each pedestrian.Meanwhile, it is possible to combine two pedestrians, who are adjacentor in close proximity, as one pedestrian. Here, and for illustrativepurposes only, the focus is on a single pedestrian crossing. Eachvehicle has Vehicle of Interest (VoI) list that includes all vehiclesthat may share useful information to the ego-vehicle.

Object information is received at 41 by the receiving vehicle. Objectinformation received by the receiving vehicle may include a distancebetween the two vehicles. For example, the exchanged information mayinclude a vertical distance and a horizontal distance between thevehicles. In this way, the receiving vehicle is able to determine thelocation of the transmitting vehicle in relation to itself. As notedabove, this information may be periodically exchanged using messagessent over a DSRC link. Other types of wireless links could also be usedby the vehicles.

Next, the location of the object is determined at 42 by the receivingvehicle. This location of the object is defined with respect to thelocation of the receiving vehicle. That is, the receiving vehicle servesas the reference frame for this second location of the object. In theexample embodiment, this second location is derived using the firstlocation of the object sent by the transmitting vehicle and the distancebetween the two vehicles as will further described below.

From the location of the object, a safety concern can be evaluated at 43by the receiving vehicle. In one embodiment, the receiving vehiclecomputes an expected collision point, D, between the object and thereceiving vehicle as seen in FIG. 5. The receiving vehicle can alsocompute a distance to collision (DTC) and/or a time to collision (TTC)as follows.

$\begin{matrix}{{DTC} = {l + d}} & (2) \\{{TTC} = \frac{DTC}{S_{A}}} & (3)\end{matrix}$

where S_(A) is the speed of vehicle A (e.g., in meters per second).These metrics are merely exemplary.

Based on the second location of the object, a safety measure can beimplemented in the receiving vehicle as indicated at 44. For example,assuming an expected collision point exists, a safety concern can beraised and a warning can be issued to the driver of the receivingvehicle. The warning can be issued at a fixed interval (e.g., 5 seconds)before an anticipated collision. The warning may a visual, audibleand/or haptic indicator. In response to a raised safety issue, thereceiving vehicle may also implement an automated preventive measure,such as automatic braking of the vehicle.

Additionally, video for the detected object is received at 45 by thereceiving vehicle. The received video can then be fused at 46 with thevideo captured by the receiving vehicle. Continuing with the example inFIG. 5, the image of the obscured pedestrian can be integrated into thevideo captured by the receiving vehicle. One technique for fusing thedata is set forth below.

After vehicle B receives a request for video streaming, vehicle B sharesonly detected pedestrian region of the image, also called Region ofInterest (RoI). Before sending the RoI to vehicle A, the RoI iscompressed into a video stream. When the vehicle receives the firstimage of the video stream, it has to determine if it is within the localcamera Horizontal Field Of Viewing (HFOV). Hence, angle ∠α is calculatedas shown in FIG. 5.

$\begin{matrix}{{{\angle a} = {\arctan \left( \frac{\tau + e}{d + l} \right)}}{where}} & (4) \\{r = {{\tan (\beta)}*l}} & (5)\end{matrix}$

Note that r might be negative if ∠β is negative. ∠β is estimated byvehicle B. A simple way to estimate an object's horizontal angle is bymeasuring the average horizontal object pixels' locations to the cameraHorizontal Field of View (HFOV) as follows:

$\begin{matrix}{{\angle \beta} = {\frac{HFOV}{2} - \left( {\frac{u}{u_{\max}}*HF0V} \right)}} & (6)\end{matrix}$

When ∠β is positive, the object is on the left side of the camera andvice versa. Now if ∠α is larger than HFOV of vehicle A, only audiblewarning is made to the driver. Otherwise the pedestrian image istransposed on the local video stream image. As shown in FIG. 6, usingthe camera pinhole model, the object is transferred from camera B imageplane to camera A image plane as follows:

$\begin{matrix}{{u_{1} = \frac{f{c_{x}\left( {X + {\Delta X}} \right)}}{Z + {\Delta Z}}}{v_{1} = \frac{f{c_{y}\left( {Y + {\Delta Y}} \right)}}{Z + {\Delta Z}}}} & (7)\end{matrix}$

ΔX, ΔY and ΔZ are the differences in coordinate between the two cameras'locations which are similar to variables shown in FIG. 5. Both variablesX and Y are estimated from camera B using:

$\begin{matrix}{{X = \frac{Z*u_{2}}{{fc}_{x}}}{Y = \frac{Z*v_{2}}{{fc}_{y}}}} & (8)\end{matrix}$

After imposing the detected object on camera A image, the fused image ispresented to the driver at 47 on a display. The process is repeateduntil vehicle B stops sharing detected object information. To avoidsharing unnecessary information, vehicle B stops sharing detected objectinformation when the object is no longer in front of the vehicle andvisible to other vehicles

$\left( {i.e.\; {r > \frac{w}{2}}} \right).$

It is important to note that share sensors information might be updatedat a different rate. As a result, time (clock) synchronization betweenthe two vehicles is necessary. It is to be understood that only therelevant steps of the processing by the image processor 24 are discussedin relation to FIG. 4, but that other software-implemented instructionsmay be needed to control and manage the overall operation of the system.

Experimental setup and results are now described for the exampleembodiment of the collision avoidance system 20. The experimental setupconsists of two vehicles (e.g., SUV and Sedan). In each vehicle, a Cohda(MK5 DSRC transceiver, Global Navigation Satellite System GNSS) and adashboard camera (DashCam) is installed. Although DSRC transceivers areequipped with GNSS, this embodiment opted to use a separate Real-TimeKinematic (RTK) GNSS because RTK-GNSS offers a high-accuracy locationestimates when compared to standalone GNSS that is used in DSRCtransvers. In these experiments, Emlid Reach RTK GNSS receiver is used,which is a low-cost off-the-shelf device. To store the collected data,all sensors on each vehicle are connected to a laptop that has RoboticOperation System (ROS) installed on it. Two vehicles' laptops areconnected via DSRC transceivers during the data collection tosynchronize laptop clocks. In addition, a bandwidth test experiment wasconducted between two vehicles to verify the available bandwidth and toemulate channel performance when conducting the experiment in the lab.

The RTK-GNSS output was set to the maximum limit of 5 Hz and the camerato 24 Frame Per second (FPS). The DSRC data rate channel was set to 6Mbps. The experiment was conducted on the Michigan State Universitycampus and surrounding areas with wide ranging speed limits up to 55kilometer-per-hour (kph). All of the experiments were conducted duringdaytime. In the first part, channel bandwidth test was collected whiledriving at a speed ranging between 0 and 55 kph; and the distancebetween the two vehicles' DSRC transceivers ranged from 5 to 100 meters.In the second part, a pedestrian pre-collision scenario was simulatedand coordinated by a test team.

In the lab setup, two ROS supported desktop PC were used and connectedwith stationary DSRC transceivers. The distance between the twotransceivers is fixed to 5 meters. To emulate the moving vehicle, basedon the road test findings, a random delay of 5 to 15 milliseconds delaywas added to the channel and the maximum channel bandwidth set to 1.8Mbps. Both PCs have core 17 processor and one PC with NVIDIA GTX 1080tiGPU. The GPU capable PC represents vehicle B while the other PCrepresents vehicle A. Proposed system components were implemented as ROSnodes. You Only Look Once (YOLO) object detection algorithm was used inthe lab experiment, such that the algorithm for pedestrian detection wastrained using Visual Object Classes (VOC) data set. Also, Motion JPEG(MJPEG) was used as the video/image encoding/decoding technique.

FIGS. 7 and 8 show a sample of DSRC bandwidth and packet delay testresults, respectively. During this sample results, the distance betweenthe two vehicles was 90 to 120 meters and at a speed of 55 kph. Theaverage bandwidth and delay were 2.85 Mbps and 34.5 ms respectively. Itwas found that DSRC equipment can carry high quality video stream withminimal delay. Similar findings are found in P. Gomes et al “MakingVehicles” Transparent Through V2V Video Streaming” IEEE Transactions onIntelligent Transportation Systems 13 (2012).

Object detection algorithm YOLO was able to process 8-10 FPS which isconsidered acceptable. However, it is possible to achieve higherprocessing using automotive oriented hardware. As discussed earlier,after a pedestrian is detected, the pedestrian distance and angle isestimated. The Region of Interest (ROI) is extracted from the originalimage and sent to the video/image encoder. The M-JPEG encoder compresseseach image individually as a JPEG image. This compression method saves asignificant amount of time compared to other advance video compressiontechniques. The average compressed image size is 3.5 KB which is muchsmaller than sharing the full image. For example, a high quality H.264video stream of 640×480 at 10 FPS requires a 1.029 Mbps while selectivesharing at 10 FPS would need only 280 Kbps. However, limit the videostreaming rate to 5 Hz similar to GNSS update rate to achieve bestaccuracy. Pedestrian distance l and <β are sent at the detection ratewhich is 8 to 10 Hz.

FIG. 9 depicts the delay at every step of operation, where overall delayis between two consecutive image fusions including the display of thefinal fused image. The average overall delay is 200 ms which is similarto the video sharing rate of 5 Hz, mainly due to the fact that the GNSSupdate is limited to 5 Hz. The fusion processes delay average is 33 msand includes the delay caused by calculation, fusion and synchronizationbetween remote and local data. Meanwhile the average channel objectdetection delays are 10 ms and 122 ms respectively. It is clear that thesum of the fusion, channel and object detection is less than overalldelay, suggesting the 200 ms delay is not possible to increase theinformation sharing rate by a) improving object detection processingrate without decreasing detection accuracy b) increasing GNSS rate.

TABLE 2 Time (seconds) Speed (m/s) DTC (m) TTC (seconds 0 8.98 20.1 2.250.2 8.94 19.1 2.18 0.4 8.65 17.99 2 0.6 8.64 16.5 1.9 0.8 8.49 15.62 1.81 8.31 14.4 1.61 1.2 7.77 12.79 1.53 1.4 7.64 11.5 1.47 1.6 7.64 10.81.42 1.8 7.10 10.1 1.41 2 6.52 9.4 1.43 2.2 6.13 9.1 1.4 2.4 5.94 8.31.9

Table 2 shows the calculations that are conducted during ourpre-collision interaction which lasted 2.4 seconds. During thatinteraction, the driver is warned about pedestrian crossing. A sample ofthe fused images is shown in FIGS. 10A-10F.

Some portions of the above description present the techniques describedherein in terms of algorithms and symbolic representations of operationson information. These algorithmic descriptions and representations arethe means used by those skilled in the data processing arts to mosteffectively convey the substance of their work to others skilled in theart. These operations, while described functionally or logically, areunderstood to be implemented by computer programs. Furthermore, it hasalso proven convenient at times to refer to these arrangements ofoperations as modules or by functional names, without loss ofgenerality.

Unless specifically stated otherwise as apparent from the abovediscussion, it is appreciated that throughout the description,discussions utilizing terms such as “processing” or “computing” or“calculating” or “determining” or “displaying” or the like, refer to theaction and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system memories orregisters or other such information storage, transmission or displaydevices.

Certain aspects of the described techniques include process steps andinstructions described herein in the form of an algorithm. It should benoted that the described process steps and instructions could beembodied in software, firmware or hardware, and when embodied insoftware, could be downloaded to reside on and be operated fromdifferent platforms used by real time network operating systems.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored on acomputer readable medium that can be accessed by the computer. Such acomputer program may be stored in a tangible computer readable storagemedium, such as, but is not limited to, any type of disk includingfloppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-onlymemories (ROMs), random access memories (RAMs), EPROMs, EEPROMs,magnetic or optical cards, application specific integrated circuits(ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The algorithms and operations presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may also be used with programs in accordancewith the teachings herein, or it may prove convenient to construct morespecialized apparatuses to perform the required method steps. Therequired structure for a variety of these systems will be apparent tothose of skill in the art, along with equivalent variations. Inaddition, the present disclosure is not described with reference to anyparticular programming language. It is appreciated that a variety ofprogramming languages may be used to implement the teachings of thepresent disclosure as described herein.

The foregoing description of the embodiments has been provided forpurposes of illustration and description. It is not intended to beexhaustive or to limit the disclosure. Individual elements or featuresof a particular embodiment are generally not limited to that particularembodiment, but, where applicable, are interchangeable and can be usedin a selected embodiment, even if not specifically shown or described.The same may also be varied in many ways. Such variations are not to beregarded as a departure from the disclosure, and all such modificationsare intended to be included within the scope of the disclosure.

What is claimed is:
 1. A method for sharing data across vehicles forimproved safety, comprising: detecting, by a processor in a transmittingvehicle, an object in an image captured by an imaging device in atransmitting vehicle; determining, by the processor in the transmittingvehicle, a first location of the object from the image, where the firstlocation of the object is defined with respect to the transmittingvehicle; sending the first location of the object from the transmittingvehicle via a dedicated short range communication link to a receivingvehicle; receiving, by a processor in the receiving vehicle, the firstlocation of the object from the transmitting vehicle; determining, bythe processor in the receiving vehicle, a vehicle location of thetransmitting vehicle with respect to the receiving vehicle; determining,by the processor in the receiving vehicle, a second location of theobject using the first location and the vehicle location, where thesecond location is defined with respect to the receiving vehicle; andimplementing a safety measure in the receiving vehicle based on thesecond location of the object.
 2. The method of claim 1 furthercomprises detecting an object in an image captured by an imaging deviceusing You Only Look Once object detection algorithm.
 3. The method ofclaim 1 wherein determining a first location of the object furthercomprises calculating a distance to the object by$l = {f_{c}\frac{R_{h}}{I_{h}}}$ where the object is a person, f_(c) isthe focal length of the imaging device, R_(h) is actual height of theperson and I_(h) is height of the person in image pixels.
 4. The methodof claim 1 further comprises sending the first location of the objectfrom the transmitting vehicle only if the distance between the objectand the transmitting vehicle is less than a predefined threshold.
 5. Themethod of claim 1 further comprises determining whether the transmittingvehicle and the receiving vehicle are traveling the same direction andsending the first location of the object to the receiving vehicle inresponse to a determination that the transmitting vehicle and thereceiving vehicle are traveling the same direction.
 6. The method ofclaim 1 further comprises computing at least one of a distance tocollision with the object or a time to collision with the object andimplementing a safety measure in the receiving vehicle in response tothe at least one of the distance to collision or the time to collisionbeing less than a threshold.
 7. The method of claim 1 whereinimplementing a safety measure includes one of issuing a warning aboutthe object to a driver of the receiving vehicle, displaying the objectto the driver of the receiving vehicle or automatically braking thereceiving vehicle.
 8. The method of claim 1 further comprises sendingimage data for the object from the transmitting vehicle via a secondarycommunication link to the receiving vehicle, where the secondarycommunication link differs from the dedicated short range communicationlink.
 9. The method of claim 8 further comprises capturing, by a cameradisposed in the receiving vehicle, video of a scene; receiving, by theprocessor in the receiving vehicle, the image data for the object fromthe transmitting vehicle; fusing, by the processor in the receivingvehicle, the image data for the object into the video; and presenting,by the processor in the receiving vehicle, the video with the image datafused therein to the driver of the receiving vehicle.
 10. A method fordetecting objects in a moving vehicle, comprising: receiving, by aprocessor in a receiving vehicle, a first location of the object fromthe image, where the first location is communicated via a wireless datalink by a transmitting vehicle and the first location of the object isdefined with respect to the transmitting vehicle; determining, by theprocessor in the receiving vehicle, a vehicle location of thetransmitting vehicle with respect to the receiving vehicle; determining,by the processor in the receiving vehicle, a second location of theobject using the first location and the vehicle location, where thesecond location is defined with respect to the receiving vehicle; andimplementing a safety measure in the receiving vehicle based on thesecond location of the object.
 11. The method of claim 10 receiving, bythe processor in the receiving vehicle, distance between thetransmitting vehicle and the receiving vehicle over a dedicated shortrange communication link.
 12. The method of claim 10 further comprisescomputing at least one of a distance to collision with the object or atime to collision with the object and implementing a safety measure inthe receiving vehicle in response to the at least one of the distance tocollision or the time to collision being less than a threshold.
 13. Themethod of claim 10 wherein implementing a safety measure includes one ofissuing a warning about the object to a driver of the receiving vehicle,displaying the object to the driver of the receiving vehicle, orautomatically braking the receiving vehicle.
 14. The method of claim 10further comprises receiving, by the processor of the receiving vehicle,image data for the object sent by the transmitting vehicle via asecondary communication link, where the secondary communication linkdiffers from the wireless data link.
 15. The method of claim 14 furthercomprises capturing, by a camera disposed in the receiving vehicle,video of a scene; fusing, by the processor in the receiving vehicle, theimage data for the object into the video; and presenting, by theprocessor in the receiving vehicle, the video with the image data fusedtherein to the driver of the receiving vehicle.
 16. A collisionavoidance system, comprising: a first camera disposed in a transmittingvehicle; a first image processor configured to receive image data fromthe first camera, the first image processor operates detect an object inthe image data and determine a first location for the object from theimage data, where the first location is defined with respect to thetransmitting vehicle; a first transceiver interfaced with the firstimage processor and operates to send the first location for the objectvia a wireless communication link to a receiving vehicle; a secondtransceiver disposed in the receiving vehicle and configured to receivethe first location of the object from the transmitting vehicle; a secondimage processor interfaced with the second transceiver, the second imageprocessor operates to determine a vehicle location of the transmittingvehicle with respect to the receiving vehicle and determine a secondlocation of the object using the first location and the vehiclelocation, where the second location is defined with respect to thereceiving vehicle.
 17. The collision avoidance system of claim 16wherein the first image processor determines the first location of theobject by calculating a distance to the object as follows$l = {f_{c}\frac{R_{h}}{I_{h}}}$ where the object is a person, f_(c) isthe focal length of the first camera, R_(h) is actual height of theperson and I_(h) is height of the person in image pixels.
 18. Thecollision avoidance system of claim 17 wherein the first image processorsends the first location of the object from the transmitting vehicleonly if the distance between the object and the transmitting vehicle isless than a predefined threshold.
 19. The collision avoidance system ofclaim 17 wherein the first image processor determines whether thetransmitting vehicle and the receiving vehicle are traveling the samedirection and sends the first location of the object to the receivingvehicle in response to a determination that the transmitting vehicle andthe receiving vehicle are traveling the same direction.
 20. Thecollision avoidance system of claim 16 wherein the first location of theobject is transmitted in accordance with Dedicated Short-rangeCommunication (DSRC) protocol.
 21. The collision avoidance system ofclaim 16 wherein the second image processor implements a safety measurein the receiving vehicle based on the second location of the object. 22.The collision avoidance system of claim 21 wherein the second imageprocessor computes at least one of a distance to collision with theobject or a time to collision with the object and implements a safetymeasure in the receiving vehicle in response to the at least one of thedistance to collision or the time to collision being less than athreshold.
 23. The collision avoidance system of claim 21 furthercomprises an automatic emergency braking system in the receivingvehicle, wherein the second image processor operates to automaticallybraking the receiving vehicle based on the second location of theobject.
 24. The collision avoidance system of claim 21 wherein the firsttransceiver send image data for the object from the transmitting vehiclevia a secondary communication link to the receiving vehicle, where thesecondary communication link differs from the wireless communicationlink.
 25. The collision avoidance system of claim 24 further comprises asecond camera in the receiving vehicle, wherein the second imageprocessor receiving video from the second camera, fuses the image datafor the object with the video, and presents the video with the imagedata to the driver of the receiving vehicle.